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ABSTRACT 

Objective: To explore the structure and content of a 
non-random sample of clinical study reports (GSRs) to 
guide clinicians and systematic reviewers. 
Search strategy: We searched public sources and 
lodged Freedom of Information requests for previously 
confidential GSRs primarily written by the industry for 
regulators. 

Selection criteria: GSRs reporting sufficient 

information for extraction ('adequate'). 

Primary outcome measures: Presence and length 

of essential elements of trial design and reporting and 

compression factor (ratio of page length for GSRs 

compared to its published counterpart in a scientific 

journal). 

Data extraction: Data were extracted on standard 
forms and crosschecked for accuracy. 
Results: We assembled a population of 78 GSRs 
(covering 90 randomised controlled trials; 144 610 
pages total) dated 1991-2011 of 14 pharmaceuticals. 
Report synopses had a median length of 5 pages, 
efficacy evaluation 13.5 pages, safety evaluation 
17 pages, attached tables 337 pages, trial protocol 
62 pages, statistical analysis plan 15 pages and 
individual efficacy and safety listings had a median 
length of 447 and 109.5 pages, respectively. While 
16 (21%) of GSRs contained completed case report 
forms, these were accessible to us in only one case 
(765 pages representing 16 individuals). Compression 
factors ranged between 1 and 8805. 
Conclusions: Clinical study reports represent a 
hitherto mostly hidden and untapped source of detailed 
and exhaustive data on each trial. They should be 
consulted by independent parties interested in a 
detailed record of a clinical trial, and should form the 
basic unit for evidence synthesis as their use is likely 
to minimise the problem of reporting bias. We cannot 
say whether our sample is representative and whether 
our conclusions are generalisable to an undefined and 
undefinable population of GSRs. 
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INTRODUCTION 

Systematic reviews are thought to provide 
one of the most robust ways to evaluate 
the effects of healthcare interventions. But 
the robustness of findings clearly rests upon 
reviewers having sufficient access to clinical 



ARTICLE SUMMARY 



Article focus 

■ What are clinical study reports (GSRs)? What do 
they contain and how long are they? 

■ Gan GSRs help address reporting biases asso- 
ciated with the published literature, and improve 
the quality of evidence synthesis? 

Key messages 

■ GSRs represent a hitherto hidden and untapped 
source of detailed randomised controlled trial 
data (mean page length: 1854 pages), increas- 
ingly becoming publicly available, and should 
form the basic unit for evidence synthesis to 
minimise the problem of reporting bias. 

■ GSRs show that numerous individuals make 
important technical contributions to the design 
conduct and reporting of each trial, but journal 
publications often fail to record these details 
resulting in a loss in individual responsibility for 
what is reported. 

■ The IGH E3 guideline to which most GSRs 
conform was published in 1995, and needs 
updating. , 

Strengths and limitations of this study 

■ We cannot say whether our sample is represen- 
tative and whether our conclusions are generalis- 
able to an undefined and undefinable population 
of GSRs. 



trial information to critically evaluate and 
reproduce the original research. Research on 
reporting bias over the last decades has 
shown that trusting the published literature 
at face value, even peer-reviewed publica- 
tions, can be fraught with difficulty — a 
problem that spans drug classes 

Following the decision by the European 
regulator, the European Medicines Agency 
(EMA) on 30 November 2010, to make avail- 
able a broad spectrum of documents related to 
medicinal products for human and veterinary 
use,^^ attention has focused on one particu- 
lar type of regulatory document — clinical study 
reports (CSRs)/^"^^ CSRs are usually written 



Doslii P, Jefferson T. BMJ Open 2013;3:e002496. doi:1 0.11 36/bmjopen-201 2-002496 



1 



Clinical study reports of randomised controlled trials 



for regulators following guidelines developed by the 
industry regulatory collaborative effort 'International 
Conference on Harmonisation of Technical Requirements 
for Registration of Pharmaceuticals for Human Use' 
(ICH). The ICH guidelines 'structure and content of 
clinical study reports' (see online supplementary appen- 
dix 1) are known by the document code 'E3'. They were 
formalised in 1995 'to assist sponsors in the development 
of a report that is complete, free from ambiguity well orga- 
nised and easy [for regulators] to review'/^ E3 has not 
been edited or changed since 1995. 

CSRs are but one category of information that is trans- 
mitted from study sponsors to regulators (figure 1), but 
are important as they contain substantially more informa- 
tion and detail on the intervention being tested than pub- 
lished versions of the same trial. The wealth of information 
may be sought with increasing frequency by researchers 
appraising single trials, entire trial programmes, or by 
those synthesising evidence. We are aware of two 
recent examples of systematic reviews of the effects of phar- 
maceuticals carried out using CSRs and other regulatory 
material.^^ One group also concluded thatjournal publi- 
cations insufficiently report clinical trials.^^ 

Despite CSRs' potential importance little is known 
about their structure and content outside of those indivi- 
duals with direct involvement in regulatory processes. 
This knowledge gap may hinder development of 
methods for fair and reliable appraisal of CSRs and 



their use in evidence synthesis. We are not aware of any 
instruments specifically designed for appraising CSRs. 
Lack of visibility may also hinder understanding of 
the complexity of the organisation and the reporting of 
clinical trials. 

We carried out an exploratory review to describe the 
structure and content of a non-random sample of CSRs. 
By describing the contents of CSRs, this research seeks 
to transform CSRs from an obscure document only 
known to regulators and industry into a more widely 
known and accessible document. Our long-term inten- 
tion is to improve the credibility of research synthesis by 
facilitating a move from the level of detail found in 
journal articles to the level of detail found in regulatory 
documents, thus guiding clinicians and other decision 
makers at all levels. 



METHODS 

We obtained CSRs from public sources, as follows: 

1. Requesting from EMA, under its Freedom of 
Information (FOI) policy, CSRs for manufacturer- 
sponsored trials of the 10 best-selling prescription- 
bound products in the USA in 2010.^^ 

2. Reusing CSRs from our own previous research (osel- 
tamivir and zanamivir).^^ 

3. Downloading CSRs openly available on the Internet. 
Search terms were not predefined, but sites searched 



Figure 1 Types of clinical trial 
data typically held within and 
transferred between three realms: 
trial sponsor, regulatory and 
public. 



Trial Sponsor Realm 



Regulatory Realm 



Public Realm 



Publication(s) if published and Poster(s) if presented 



Documents shared with 
regulators: 

• Trial Protocol and 
amendments 

• Clinical Study Report 

• Certificates of 
Analysis 

• Case Report Forms 

• Electronic datasets of 
individual participant 
data 



Documents kept 
internally: 

• Internal 
correspondence 

• Meeting minutes 

• Marketing 
Assessments 




4lh 



EMA 

EMA-Sponsor 
correspondence 
Trial Protocol and 
amendments 
Clinical Study Report but 
often lacks all appendices 
Case Report Forms 
(usually only a small 
subset) 
ertificates of Analysis 



FDA 

FDA-Sponsor 
correspondence 
Trial Protocol and 
amendments 
Clinical Study Report 
Case Report Forms 
(usually only a small 
subset) 

Certificates of Analysis 
Electronic datasets of _ 
individual participant data 



Since Nov 2010, 
bcuments held by 
EMA with possible 
redactions 



FDA Approval 
History, Letters, 
Reviews, and 
Related Documents 
(Drugs@FDA) with 
redactions 
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included Google (http://www.google.com), the Drug 
Industry Document Archive (http:/ / dida. library. ucsf. 
edu/) and Institut fur Qualitat und Wirtschaftlichkeit 
im Gesundheitswesen (Institute for Quality and 
Efficiency in Healthcare) (IQWiG)'s library of rebox- 
etine studies (https://www.iqwig.de/information- 
on-studies-of-reboxetine.980.en.html) . 

4. Corresponding with one researcher who obtained 
GSRs through an FOI request to Food and Drug 
Administration (FDA) (epoetin alfa). 

5. Requesting manufacturers fill any gaps in the com- 
pleteness of reports that we believe are legally 
required to be publicly available (paroxetine). 

To create as broad a database as possible, we did not 
apply restrictions in drug type or family or sponsor. We did 
not submit requests under the Freedom of Information 
Act to the FDA, because such requests can take years to be 
fulfilled and — if fulfilled — may be heavily redacted.^^ 

We did not draw a random sample of GSRs as there is 
no known sampling frame. No one knows how many 
reports have been written by intervention category as 
there is no central register of GSRs. Through familiarity 
with GSRs for oseltamivir and zanamivir, which were 
included in one of our Cochrane reviews, we devel- 
oped and piloted a data extraction sheet designed to 
capture the salient characteristics of GSRs. We created a 
list of around 40 potential sections we expected to find, 
generated directly from elements specified in E3. For 
each element in the list, we checked whether the 
obtained GSRs included that section (confirmed either 
by direct identification of the section or an indication 
the section existed based on the GSRs' table of con- 
tents), whether we had access to it and its page length. 
Because of previous difficulties we had accessing GSRs 
appendices, we also recorded whether sections were 
listed as appendices or not. The page length was calcu- 
lated either by directly counting the pages or by estimat- 
ing their size from the table of contents of each report, 
and was used as a crude proxy for the level of detail 
available. The page lengths were rounded up to the next 
integer, and were summarised by reporting medians and 
ranges. We also included questions relating to trial regis- 
tration and authorship. Our (blank) data extraction 
sheet is in online supplementary appendix 2. 

All variables from GSRs were first extracted in single. 
We subsequently audited each other's extractions, check- 
ing the accurateness of the information. We chose to 
present elements analogous with those that typically 
appear in trials reported in scientific journals including 
the study synopsis (a brief summary of the study), the 
study protocol (written prospectively, describing the study 
methods), efficacy and safety evaluations (a narrative 
summary of the efficacy and safety results of the study, 
including tables and figures), as well as attached tables. 
We also included elements rarely found in journal publi- 
cations: sample (blank) and completed case report forms 
(GRFs are paper or electronic forms designed to capture 
prespecified efficacy and safety related information 



for each study participant), the statistical analysis plan (a 
prospectively written narrative and/or statistical code 
indicating how trial data will be analysed) and individual 
participant efficacy and safety listings. The corresponding 
E3 section numbers are listed in table 2. Disagreements 
were resolved by discussion. 

Our uncorrected (original) and corrected extraction 
sheets as well as audit records are available upon request 
from the corresponding author. 

We calculated a compression factor for published 
trials which we defined as the ratio of GSR page length 
compared to the page length of the same trial as pub- 
lished in scientific journals. The objective of this metric 
was to convey a rough sense of how much information 
present in GSRs may be condensed ('compressed') in 
short journal publications, in consideration of GSRs' far 
greater length and level of detail. The size (page 
length) reflects the level of detail as well as the presence 
of many elements such as protocols and their amend- 
ments, randomisation lists, statistical analysis plans, certi- 
ficates of analysis and extra data on subpopulations. We 
have demonstrated^^ that these elements are essential 
for understanding and appraising a trial. The compres- 
sion factor is a crude measure of how much is com- 
pressed or simply left out of each publication which will 
affect the reliability of the appraisal and interpretation 
of trials. Trial publications were searched for in multiple 
sources — clinical trial registers, published systematic 
reviews and correspondence with sponsors. Because in 
most cases we could not access all parts of all GSRs (and 
therefore do not know their complete page length), we 
calculated 'conservative' compression factors as well as 
'realistic' compression factors. 'Conservative' compres- 
sion factors were calculated on a trial by trial basis using 
the total number of pages in GSRs available to us 
divided by the length of journal reports for that same 
trial, whereas 'realistic' compression factors were based 
on the true total page length of the GSRs. 

RESULTS 

We identified 84 documents believed to be GSRs for 14 
compounds. These covered therapeutic and biological 
interventions including antipsychotics, antidepressants, 
antivirals, natural antiarthritics, anti-inflammatory 
agents, pandemic influenza vaccines, statins, erythro- 
poietins and antiplatelet compounds. We included 
English-language summaries of two Japanese oseltamivir 
studies (JV15823 and JV15824) as they had been pre- 
sented to EMA in this form. We excluded documents 
which were sections of GSRs that nonetheless contained 
insufficient information to understand the overall 
content of the GSRs (olanzapine FID-LG-HGAV, 
FID-MG-HGAJ and FID-MG-HGAO) and three docu- 
ments which we had originally classified as GSRs but 
were not (reboxetine 14, 22 and 37). This left us with 78 
GSRs (144 610 pages) (figure 2). The median pages 
obtained per GSR was 644 (range 9-15 440). Only 4 of 
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Figure 2 Study flow. 



Possible CSRs identified 
through public access 
sources (n = 45) 



CSRs identified through 
our prior systematic 
review (n = 26) 



CSRs requested and received 
from European (n = 10) and 
US (n=l) regulators 



Possible CSRs after duplicates removed 
(n = 84) 




NotaCSR(n = 3) 

Insufficient information to 
understand the overall 
content oftheCSR (n=3) 



CSRs with sufficient information for extracting 
(n = 78) 



CSRs extracted 
(n = 78) 



78 CSRs (reboxetine 8, 16, 17 and 91) were written 
prior to 30 November 1995 when ICH E3 was approved. 
Table 1 summarises the pharmaceutical, manufacturer, 
date and provenance of the CSRs in our review EMA 
reported not holding studies for esomeprazole magne- 
sium (Nexium), Advair Diskus, quetiapine fumarate 
(Seroquel), montelukast sodium (Singulair), epoetin 
alfa (Epogen) and simvastatin. 

All of the 78 included CSRs contained a synopsis 
(median page length 5 pages). The efficacy evaluation 
was identifiable and directly accessible in 76 (97%; 
median length 13.5 pages) and safety in 77 (99%; 
median length 17 pages). The attached tables were like- 
wise present in 63 (81%) CSRs, with a median of 337 
pages long (range 1-3665). Seventy-three CSRs (94%) 
reported including the study protocol. In the 40 proto- 
cols we could access, the median page length was 62. We 
found blank CRFs included in 68 (87%) CSRs. Of the 33 
blank CRFs we could directly access, the median length 
was 133 pages (range 14-981). For completed CRFs, 16 
(21%) reports made direct mention of a section on com- 
pleted CRFs, but we had access to completed CRFs in 
only 1 case (Arthronat; length 765 pages). 

Fifty-five (71%) of 78 included CSRs included a statis- 
tical analysis plan in some form. Of those for which we 
could directly access the content (n=37) , the median page 
length was 15 (range 3-85). Individual efficacy and safety 
listings were included in 53 (69%) and 62 (81%) CSRs, 
respectively. The median page length was 447 (range 15- 
21 698) for efficacy and 109.5 (range 2-10 954) for safety. 



A summary is presented in table 2. 

All trial reports in our review were sponsored by the 
pharmaceutical industry. 

Median conservative compression factors ranged 
between 1 and 1221. The realistic compression factors 
calculated for the Arthronat, paroxetine and 
Clopidogrel versus Aspirin In Patients at Risk of 
Ischaemic Events (CARRIE) trials were 379, 1021 and 
8805, respectively (table 3). 

DISCUSSION 

We collected and described a sizeable number of CSRs 
written in the last two decades. All CSRs contained a table 
of contents (as specified in E3 section 3); this, together 
with optical character recognition (to enable searching the 
full text of the scanned documents) and the occasional 
need to combine multiple files to create a single docu- 
ment, substantially improved the ease of navigating CSRs. 

Despite the size of our non-random sample, it is 
unclear whether our conclusions are generalisable to all 
other CSRs. This is because we have extremely limited 
knowledge about the total population of CSRs in regula- 
tors' and sponsors' possession. Nevertheless, within our 
sample spanning different manufacturers, therapeutic 
classes and times, we found that the structure of CSRs 
was, within different house styles of presentation, strik- 
ingly similar, probably owing to the guidance by ICH 
E3.^^ This suggests that the structure and content of 
other CSRs is likely to be similar. 
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Table 1 Pharmaceutical, trials, producers, dates and sources of CSRs in the review 






CD 
C/5 

o 

—1 

i 


Pharmaceutical and 

number (n) of assessed Date of 
trial documents Trial IDs Manufacturer CSRs 


Provenance in our study 





Aripiprazole (Ability) n=1 CN1368135 



Arthronat n=1 
Atorvastatin (Lipitor) n=1 
Clopidogrel (Plavix) n=5 

Epoetin alfa (Epogen) n=1 
H5N1 influenza vaccine 
n=1 

H5N1 influenza vaccines 
n=2 

Olanzapine (Zyprexa) n=3 
Oseltamivir (Tamiflu) n=19 




Paroxetine (Paxil, Aropax, 
Pexeva, Seroxat, 
Sereupin) n=9 



MA-CT-1 0-002 
981-080 

CURE, CLARITY, C0MMIT-CCS2, CAPRIE, 

PICOLO 

930 107 

H5N1-008, H5N1-011 EXT 008 
V87P1, V87P6 

F1D-LC-HGAV*, F1 D-MC-HGAO*, 
F1D-MC-HGAJ* 

JV15823, JV15824, M76001, NP15757, 
NV1 6871, WP1 6263, WV15670, WV15671, 
WV15673, WV15697, WV15707, WV15708, 
WV15730, WV15758, WV15759, WV15871, 
WV15799, WV15812, WV15872, WV15819, 
WV15876, WV15978, WV15825, WV16193 
329, 377, 453, 511, 676, 701, 704, 715, 716 



Quetiapine (Seroquel) n=7 015, 041, 049, 125, 126, 127, 135 



Reboxetine (Edronax, 
Norebox, Prolift, Solvex, 
Davedax, Vestra) n=24 



8, 9, 13, 14*, 15, 16, 17, 22*, 32, 32a, 34, 
35, 37*, 43, 45, 46, 47, 49, 50, 52, 71, 83, 
91, 96 



Rofecoxib (Vioxx) n=1 78 

Zanamivir (Relenza) n=9 NAI30009, NAI300010, NAIA2005, 

NAIA3002, NAIA3005, NAIB2005, 
NAIB2007, NAIB3001, NAIB3002 



Bristol-Myers 
Squibb 
Rowtasha 
Pfizer 

Bristol-Myers 
Squibb 
Amgen 
GSK 

Novartis 

Eli Lilly 

Roche 



GSK 

AstraZeneca 
Pfizer 



Merck 
GSK 



2007 

2011 
1999 

1997-2007 

1996 
2006 

2008-2009 

1995t 

1999-2004 



1998-2002 

1996-2007 
1991-2009 



2003 

1998-1999 



Freedom of Information request to the EMA 

Manufacturer website http://arthronat.com/clinical-study.php 
Freedom of Information request to the EMA 
Freedom of Information request to the EMA 

Freedom of Information request to the FDA 
Freedom of Information request to the EMA 

Freedom of Information request to the EMA 

Litigation http://www.furiousseasons.com/zyprexadocs.html 

Documents obtained as part of previous Cochrane review"*^ 




Litigation (2004 legal settlement mandated release of clinical 
study reports on manufacturer's website of 9 studies on 
paediatric and adolescent patients) http://www.gsk.com/ 
med ia/paroxeti ne . htm 

Litigaton http://psychrights.org/research/Digest/NLPs/ 

Seroquel/UnsealedSeroquelStudies/ 

Health technology assessment website (The German IQWiG 

obtained CSRs as part of its health technology assessment 

work) https://www.iqwig.de/ 

information-on-studies-of-reboxetine.980.en.html 

Litigation http://dida.library.ucsf.edu/ 

Documents obtained as part of previous Cochrane review ""^ 



*Subsequently excluded because of insufficient documentation. 
tH1D-MC-HGA0 clinical study report date unknown. 

CSRs, clinical study reports; EMA, European Medicines Agency; FDA, Food and Drug Administration; GSK, GlaxoSmithKline; IQWiG, Institute for Quality and Efficiency in Healthcare; 
CAPRIE, Clopidogrel versus Aspirin In Patients at Risk of Ischaemic Events. 
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Table 2 Key characteristics of the clinical study reports in the review ^^^^^^ 




Presence 


Length 






CSRs including 


CSRs with section 


Median length 


Section of CSR (corresponding section of E3) 


section, n 


length available, n 


(range), pages 


Synopsis (E3 section 2) 


"70 /H r\r\o/ \ 
lo (lUUVo) 


"70 
10 


0 (1— 1b) 


Efficacy evaluation (E3 section 11) 


"7C /n"70/ \ 

/b (y/ 7o) 


-7-7 
/ / 


HOC /O H 00\ 

lo.b ytL—xoii) 


Safety evaluation (E3 section 12) 


-7-7 /noo/ \ 

77 (yy%) 


CO 

00 


H "7 /O H 00\ 

17 loo) 


Attached tables not in report text (E3 section 14) 


00 /Q i 0/ \ 
DO \p\ /o) 


10 


00/ (1— obbb) 


Protocol (E3 section 16.1.1) 


73 (94%) 


41 


62 (21-139) 


Blank CRFs (E3 section 16.1.2) 


68 (87%) 


33 


133 (14-981) 


Statistical analysis plan (E3 section 16.1.9) 


55 (71%) 


37 


15 (3-85) 


Individual participant efficacy listings (E3 section 16.2.6) 


53 (69%) 


19 


447 (15-21698) 


Individual participant safety listings (E3 section 16.2.7) 


62 (81%) 


26 


109.5 (2-10954) 


Completed CRFs (E3 section 16.3.2) 


16(21%) 


1 


765 


CRFs, case report forms; CSRs, clinical study reports; E3, ICH E3. 



The future basic currency of research synthesis? 

The median length of 644 pages for reports in this 
study, as well as CSRs' routine inclusion of trials' proto- 
col, statistical analysis plans and blank case report forms, 
strongly suggests that CSRs are the most detailed and 
complete, integrated form of reporting of the design, 
conduct and results of clinical trials. In a study that dir- 
ectly compared the adequacy of reporting between 
journal articles and CSRs, the authors found that com- 
plete information regarding greater than 40% of 
methods items were only available in CSRs.^^ The level 
of detail found in CSRs thus far surpass the level of 
detail available in journal publications, and as such they 
are prime candidates for the next basic currency of evi- 
dence synthesis and appraisal of a trial. Given the EMA's 
new policy of making such documents publicly available, 
access to these documents is now relatively straightfor- 
ward.^^ However, including CSRs in systematic reviews is 
labor-intensive, given their size and complexity. 

Accessing complete CSRs 

Although CSRs may trump other forms of trial reporting 
in the public domain (such as conference abstracts or 
journal publications), serious limitations remain. Despite 
obtaining 144 610 pages for 78 CSRs, in almost all 
instances, we lacked complete access to the CSRs' 
numerous appendices. Even for the sole complete CSR 
we obtained (Arthronat MA-CT-1 0-002), case report 
forms were provided for only 20% of participants. The 
Arthronat text does not provide a reason for this omis- 
sion, but it reflects the vagueness of the relevant section 
of the E3 guidance (16.3.2) which does not define 
'other CRFs submitted'. Also, we could only access the 
original trial protocol in 40 (51%) of 78 CSRs obtained. 
This is important because trial protocols, written prior 
to patient enrollment in a trial, are an important way to 
guard against reporting biases. ^'^ 

We could obtain individual patient listings in only a 
minority of cases despite confirming their inclusion in the 
majority of CSRs (table 2). This may be a significant 



limitation, as the E3 specifies that 'the report with its 
appendices should also provide enough individual patient 
data, including the demographic and baseline data, and 
details of analytical methods, to allow replication of the crit- 
ical analyses. '^^ Unavailability was possibly owing to the fact 
that EMA allows manufacturers to submit CSRs omitting a 
number of appendices including individual patient data 
and case report forms (which EMA states should be avail- 
able within 48 h, if requested) P In the case of oseltamivir, 
the primary drug analysed in a Cochrane review we con- 
ducted,^^ the manufacturer refused to share with us report 
appendices not submitted to EMA,^^ and EMA declined 
requesting them on our behalf. Although FDA likely pos- 
sesses more complete CSRs and patient4evel data, it histor- 
ically has treated such data as trade secret and/or 
confidential.^^"^^ EMA is therefore at present the only 



Table 3 Conservative and realistic compression factors 

k Studies Mean 

published in compression 
Pharmaceutical journals, n factor (range) 

Conservative compression factors 



Aripiprazole 


1 


672 


Clopidogrel 


5 


11 (4-19) ■ 


Epoetin alfa 


1 


41 


Fluad 


2 


488 (367-609) 


GSK H5N1 vaccine 


1 


19 


Oseltamivir 


12 


195 (1-1221) 


Quetiapine 


2 


578 (352-803) 


Reboxetine 


5 


88 (9-245) 


Zanamivir 


8 


54 (28-92) 


Realistic compression factors 




Arthronat* 


1 


379 m 


Clopidogrel 


1 


8805 • 


Paroxetine 


9 


1021 (50-5473) 



A ratio of clinical study report page length to corresponding journal 
publication page length. 

*The Arthronat trial has not yet been published. Compression 
factor calculation is based on the page length of a draft 
manuscript 'to be published soon,' according to Arthronat.com. ^ 
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reliable source of obtaining CSRs. As such, despite 
European regulators' progressive stance — announcing that 
'clinical trial data should not be considered commercial 
confidential information'^^ — the completeness gap is 
unlikely to be filled any time soon. 

Another significant limitation is that CSRs are only 
written for therapeutic, prophylactic or diagnostic 
agents, and therefore inadequacies remain in evidence 
synthesis of other types of interventions such as surgical 
or behavioural interventions. 

Individual participant listings 

Individual participant listings — which identify partici- 
pants by a unique ID — were accessible in 29 of the 78 
CSRs we reviewed. But these data are difficult to analyse 
because they are presented as database printouts rather 
than in the original computer data files. This is under- 
standable considering that CSRs are a written/archival 
format, but because EMA does not accept SAS format 
data files,^^ the industry standard, third-party access to 
databases of patient-level data remains elusive. We see 
no compelling reason why all regulators should not 
request these from sponsors and make them publicly 
available. Whether availability of individual listings 
and CRFs, with its attendant laborious analysis, would 
increase our understanding of the trial and its results is 
unclear. But there is at least one case where the reanaly- 
sis of CRFs added invaluable knowledge to that already 
available in CSRs.^'^ 

The public-private debate 

One manufacturer has claimed that the non-release of 
case report forms is motivated by concerns over protect- 
ing participants' confidentiality.^^ Nothing we have seen 
so far corroborates this claim, however an ongoing EMA 
working group is specifically discussing issues related to 
protecting participants' confidentiality. Based on current 
document releases and position statements, however, it 
appears that EMA has deemed case report forms and 
individual patient listings to be, in principle, releasable 
in their entirety (after a preliminary review). 
Furthermore, individual patient listings are intended to 
duplicate information contained in filled case report 
forms. The release of case report forms would ensure 
the accuracy of individual patient listings with little add- 
itional risk to patient confidentiality. Moreover, extra 
checks such as registration of protocols by bona fide 
research groups could deter any inappropriate use. We 
also believe that the sheer bulk of the forms act as a 
deterrent against malice. 

Size matters 

Our range of compression factors shows the scale of 
selection and synthesis which must (consciously or 
unconsciously) occur in the process of transforming 
CSRs into journal-length articles. We found a strong 
resemblance in detail, page length, structure and 
purpose between the short synopsis section of CSRs and 



reports of trials as published in scientific journals. In 
some cases essential items of information such as the 
trial protocol and its subsequent amendments are simply 
not included in journal articles or are replaced by 
methods written post facto. In other cases of items essen- 
tial for the interpretations of the trial results (such as 
the statistical analysis plan) , tens of pages are reduced to 
a paragraph on sample size calculation in the journal 
report, underscoring the lack of detail (and its attendant 
problems) common to public forms of trial reporting. 
For example, the ratio of words in the protocol of the 
CSR for aripiprazole CN138135 to the methods section 
for published journal article of the same trial is 30.5 
(53 713 words in the CSR protocol vs 1763 words in the 
journal article). For the oseltamivir WP16263 trial, the 
ratio was 22.7 (26 761 words in the CSR protocol and 
amendments vs 1177 words in the journal article). 

This compression of information also occurs in databases 
not restricted by length, such as ClinicalTrials.gov.^^ 

Our study raises the question of why the medical com- 
munity has accepted the low (summary, aggregate) level 
of detail found in most peer-reviewed journal publica- 
tions compared with the depth of detail available in 
CSRs. European regulators recently noted: 'documents 
that provide critical information on a study, such as the 
protocol (16.1.1), statistical methods (16.1.9), list of 
investigators and study sites and sample case report 
forms, would always be needed by reviewers assessing a 
study. '^^ Why have those outside of the regulatory world 
tolerated journal publications lacking such details? 

One possibility may be that while the clinical trial 
enterprise has changed dramatically in the last half 
century, the scientific journal publication model has not. 
Since the 1950s, there have been considerable transfor- 
mations in the political economy of clinical trials driven 
by the increasingly commercialised and global nature of 
the pharmaceutical industry, the rise in academic- 
industry 'partnerships' in medicine and increased com- 
munication among regulators. It is now common to find 
trials with study centres scattered around the globe. This 
increasing complexity and the need to provide an audit 
record is reflected in the comprehensive tomes docu- 
menting the trials — CSRs — but trial reporting in scien- 
tific journals remains limited to summary and aggregate 
details. It should be noted, however, that many journals 
now have websites which enable them to make available 
extended content beyond what traditionally appears in 
the printed journal. 

Authorship or contributorship? 

Examination of CSRs revealed scores of important tech- 
nical contributions to the design, conduct and reporting 
of each trial. These included contributions from data- 
base programmers, records officers and CSR writers, 
often invisible in the published journal article. In some 
cases, we found no mention in CSRs of individuals 
who figured as authors of subsequent published trial 
reports while individuals named as CSR authors went 
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unacknowledged in journal publications. Current 
International Committee of Medical Journal Editors 
(ICMJE) guidelines on authorship and contributorship 
are largely focused on ensuring those placed on by-lines 
deserve to be authors. But the guidelines also suggest 
that 'all contributors who do not meet the criteria for 
authorship should be listed in an acknowledgements 
section '.^^ Given the complexity of clinical trials, the 
ICMJE should call for itemised contributorship: the 
names of all contributors to be specified along with 
their role in the design, conduct, analysis or reporting 
of the trial. If the contribution to the trial of most 
people goes unrecorded, so does their individual 
responsibility for what is produced. Itemised contribu- 
torship records, to all phases of a trial, could be piloted 
in trial registers. 

E3 guidance 

The E3 guideline set an excellent standard, but it needs 
formal updating and further development. For example, 
there should be a self-standing set of definitions for 
terms such as 'CRFs' and 'other CRFs submitted' (section 
16.3.2) and a description of how a particular trial fits 
within a sponsor's trial programme of pharmaceutical 
development. Apparently forgotten items such as certifi- 
cates of analysis (describing the appearance and content 
of the interventions being tested) and post-1995 details 
such as trial registration numbers should be mentioned. 

We hope our review has given CSRs what they have 
lacked so far: visibility. CSRs represent a largely 
untapped source of detailed data that we believe can 
serve as a means of addressing the ravages of reporting 
bias in all its forms, leading to a more accurate under- 
standing of the effects of medicines. 
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